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Background: Most of the studies in academic 
journals use p values to represent statistical 
significance. However, this is not a good 
indicator of practical significance. Although 
confidence intervals provide information about 
the precision of point estimation, they are, 
unfortunately, rarely used. The infrequent use of 
confidence intervals might be due to estimation 
difficulties for some statistics. The bootstrap 
method enables researchers to calculate 
confidence intervals for any statistics. Bootstrap 
resampling is an effective method of computing 
confidence intervals for nearly any estimate, but 
it is not very commonly used. This may be 
because this method is not well known or people 
may think that it is complex to calculate. On the 
other hand, researchers may not be familiar with 
R and be unable to write proper codes. 


Purpose: The purpose of this study is to present the steps in the bootstrap resampling method to 
calculate confidence intervals using R. It is aimed toward guiding graduate students and researchers 
who wish to implement this method. Computations of bootstrapped confidence interval for mean, 
median and Cronbach's alpha coefficients were explained with the R syntax step-by-step. Moreover, 
traditional and bootstrapped confidence intervals and bootstrapped methods were compared in order 
to guide researchers. Main Argument and Conclusions: With the help of statistical software today it 
is easy to compute confidence intervals for almost any statistics of interest. In this study R syntax were 
used as an example so that beginners can use R to compute confidence intervals. Results showed that 
traditional and bootstrapped confidence intervals have very similar results for normally distributed 
data sets. Moreover different bootstrapped methods produce different results with skewed data sets. 
This is because bias corrected and accelerated interval methods are suggested for use with skewed 
data sets. Implications for Research and Practice: R codes presented in this study guide researchers 
and graduate students while computing bootstrap confidence intervals. Furthermore findings about 
the comparison of bootstrap methods help researchers choose the most appropriate bootstrap 
methods. Results and the main argument of this study may encourage researchers to compute 
bootstrap confidence intervals in their studies. 


© 2017 Ani Publishing Ltd. All rights reserved 


Ankara University, Faculty of Educational Science, Department of Measurement and Evaluation, 
dehadogan@gmail.com 











C. Delia DOG AN / Eurasian Journal of Educational Research 68 (2017) 1-17 


2 


Introduction 

The p value is the probability under the assumption that there is no true effect or 
no true difference of collecting data that shows a difference equal to or more extreme 
than what it is actually observed (Reinhart, 2015). Hypothesis testing uses p value to 
get statistical significance. It is the most widely reported statistic in academic 
journals. M. Marshall et al. (2000) highlighted that significance of testing is reported 
in 97% of the research papers in experimental psychology research journals. 
However the p value has some limitations and it is not an indicator of practical 
significance. It is well known that confidence intervals provide more information 
than p values (Haukas & Lewis, 2005). Effect sizes, confidence intervals, and 
confidence intervals of effect sizes are indicators of practical significance (Banjanovic 
& Osborne, 2015). Editors of many scientific journals require the use of confidence 
intervals (Cooper, Wears & Schriger, 2003). Moreover APA's Publication Manual 
(2001) highlighted the importance of calculating and reporting confidence intervals 
and effect sizes in academic research. 

Unfortunately confidence intervals are rarely reported in academic papers. 
Reinhart (2015) stated the reasons for this as follows: 

It's best to do statistics the same way everyone else does , or else the 
reviewers might reject your paper. Or maybe the widespread 
confusion about p values obscures the benefits of confidence intervals. 

Or the overemphasis on hypothesis testing in statistics courses means 
most scientists don't know how to calculate and use confidence 
intervals. 

According to Banjanovic and Osborne (2015) the infrequent use of confidence 
intervals is due to estimation difficulties for some statistics. Some statistics may 
require multi-step formulas with assumptions that might not always be viable for 
calculating confidence intervals. 

The bootstrap method enables researchers to calculate confidence intervals for 
any statistics regardless of the data's underlying distribution. The empirical 
bootstrap was introduced in 1979 (Efron 1988), but it was feasible to implement it 
without modern computing power. However, computers and statistical software 
have improved a lot, and today it is possible to calculate confidence intervals using 
the bootstrap method. Moreover free and open source R software enables researchers 
to write their own syntax to calculate confidence intervals for various statistics. 


Bootstrapping 

Briefly, bootstrap methods are resampling techniques for assessing uncertainty. 
In a broad sense the bootstrap is a widely applicable and extremely powerful 
statistical tool that can be used to quantify the uncertainty associated with a given 
estimator or statistical learning method (James et al., 2014). Bootstrap resampling is a 
method of computing confidence intervals for nearly any estimate. In most studies 
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the researchers begin with the population and take a sample from the population and 
run an analysis on that sample. In bootstrap resampling additional sub-samplings 
and replications are implemented on the original sample. In other words in the 
beginning of the process thousands of "bootstrapped resamples" are generated from 
the original sampling using random sampling with replacements. Then the 
designated statistic (mean, median, regression, Cronbach's alpha coefficient, etc.) is 
replicated in each of these resamples. Therefore, researchers may get thousands of 
estimates on the designated statistics. Distribution of those estimates is called 
"bootstrap distributions". The bootstrap distribution may be used to estimate more 
robust empirical confidence intervals. In bootstrap sampling the number of 
replications is very important. Diciccio and Efron (1996) highlight the importance of 
using at least 2000 replications while conducting bootstrap resampling. A schematic 
description of the steps for estimating confidence intervals using bootstrap formed 
by Haukoos & Lewis (2005) is shown in Figure 1. 


Bootstrap 

Samples 


Bootstrap 
Replicaiioris of 
Statistic 



Bootstrap Estimate 
of Confidence 
Interval 


Figure 1: Description of the steps in bootstrapping. 


Methods of Bootstrapping 

There are different methods for estimating confidence intervals from a 
bootstrapped distribution. The most frequently used methods are: 

• The normal interval method 
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• The percentile interval method 

• The basic interval method 

• The bias corrected and accelerated interval method. 

The normal interval method computes an approximate standard error using 
bootstrap distribution (sampling distributions resulting from bootstrap resamples). 
Then Z distribution is used to get the confidence interval. The percentile interval 
method uses a frequency histogram of the specific statistics computed from bootstrap 
samplings. The 2.5 and 97.5 percentiles constitute the limits of 95% confidence 
intervals (Haukoos & Lewis, 2005). The percentile interval method makes no 
adjustment while the student interval method corrects each statistic by its associated 
standard error and converts the distribution to studentized distribution. Then the 
confidence intervals are found at the 0.025 and 0.975 quantiles as done in the 
percentile interval method. The bias corrected and accelerated interval method 
corrects the distribution for bias and acceleration. This method adjusts the 
distribution based on two coefficient called "bias correction" and "acceleration". The 
bias correction adjusts for the skewness in bootstrap distribution; it will be zero when 
bootstrap sampling is perfectly symmetric. On the other hand, coefficients of 
acceleration do corrections for non-constant variances within the resampled data set 
(Efron, 1988). Then confidence intervals are found at the 0.025 and 0.975 quantiles of 
the corrected distribution. The basic interval method corrects the distribution for bias 
and detects the lower and upper bounds which cover the desired confidence interval 
(Banjanovic & Osborne, 2015). Each bootstrapping method has advantages and 
disadvantages, and it is important to use the more appropriate method when 
computing confidence intervals for the statistic of interest. 

Unfortunately, it is not very common to use the bootstrap method to calculate 
confidence intervals. This may be because they are not well known or people may 
think that it is complex to calculate. There is statistical software that enables users to 
compute confidence intervals using bootstrap methods. R is one such software. It is a 
language and environment for statistical computing and graphics. Because it is free 
and open-sourced, R has become popular recently for statistical data analysis. 
Moreover R syntax may seem complicated for people and this may dissuade them 
from use bootstrapping methods to calculate confidence intervals. Furthermore, 
comparisons of bootstrapping methods may guide researchers while deciding the 
methods to be used. 

The purpose of this study is to present the steps in the bootstrap resampling 
method to calculate confidence intervals using R syntax. It is aimed to guide 
graduate students and researchers who wish to implement bootstrap resampling 
using R programming language. Computation of bootstrapped confidence intervals 
for mean, median and Cronbach's alpha coefficient were explained step-by-step 
using the R syntax. Moreover, some comparisons have been made. Traditional and 
bootstrapped confidence intervals were compared while computing mean for 
normally distributed data and median for normally distributed and skewed data 
sets. 
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In this study R codes were written on R consoles and directly copied and pasted 
on this manuscript. This is why R codes have different letter characters and sizes 
from the other texts in the paper; so many R codes were presented in the paper, they 
were not defined as figures. 

Traditional Confidence Interval for Mean 

Traditionally confidence intervals are computed using the formula x ± 1.96 * 
(~j=) where x is the mean and s is the standard deviation and n is the sample size. In 
this part at first a pedagogical example is presented to compare traditional 
confidence intervals and bootstrap confidence intervals. We will first generate 
random data with 100,000 observations. This data set is viewed as population. The 
population has random normal distribution (Mu=60, Sigma=7) with 100,000 
observations. 

> population<-rnorm (100000 r 60 r 7) 

> str (population) 

nun [1:100000] 64.5 65.2 51.9 61.5 50.5 ... 


Next we take six random samples of 50 observations from the population in order 
to compute traditional confidence intervals. 

> list<-vector("list™,6) 

> for £ i in 1:6) [list [ [i| ]<-3anple(population, 50 ) } 

> 3tr (list) 

List of 6 

$ : nun [1:50] 51.3 63.5 74.7 60.5 62.3 ... 

$ : nun [1:50] 57.5 43.7 70.2 74.3 57 ... 

$ : nun [1:50] 52.9 50.3 63.2 56.7 62.3 ... 

$ : nun [1:50] 57 55.3 62.7 69.4 62 ... 

$ : nun [1:50] 59.5 56.6 59.1 55.6 53.7 ... 

$ : nun [1:50] 59.9 54.4 52.3 57.5 52.5 ... 


The next step is to write a simple function to calculate confidence intervals for the 
six samples taken randomly from the population. 

> conf . intervaK-function (x) { nean=nean (x) ; sd<-sd(x); n<-length (x) 

+ seC-1.96*(sd/(n**.5>); ci<-c(mean=niean r lowerci=mean-se r upperci=mean+se) 
# return(ci ) } 

> conf.interval(list [[1]] ) 

mean lowered upperci 
59.35779 57.66439 62.05119 


Now it is easy to compute 95% confidence intervals for the samples randomly 
taken from the population. We can round the results to two digits using the round () 
function. 
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> round(conf.interval(Xist[ [ 1 ]] ) , 2) 
mean lowered upperci 
59.36 57.66 62.05 


We can also write another function to take six random samples from the 
population. It would be better to define set.seed () for each sample. We will use the 
same samples to generate six different bootstrap distributions so we can compare 
confidence intervals. 

> conf.intervall 

function(x,seed) { set.seed(seed) 

samp1e<-samp1e(x, 50); meam<-mean (sample);sd<-sd(sample) 
n<-length(sample); se<-l.96*(sd/(n**.5)) 

ci<-round(c(mean=mean,lowerci=mean-se ,upperci=mean4se), 2 ) 
return(ci ) } 

conf . intervall (population,. seed=10) 
mean lowerci upperci 
61.33 59.76 62.90 

conf .intervall(population,seed=20) 
mean lowerci upperci 
60.46 53.27 62.65 

conf .intervall(population,3eed=30) 
mean lowerci upperci 
59.73 57.55 61.91 

conf.intervall(population,seed=40) 
mean lowerci upperci 
59.21 57.33 61.09 

conf .intervall(population,seed=50) 
mean lowerci upperci 
61.71 59.53 63.39 

conf .intervall(population,seed=6 0) 
mean lowerci upperci 
61.24 59.03 63.45 


Now we can compute 95% confidence intervals of six samples randomly taken 
from the population. We defined set.seeds as 10, 20, 30, 40, 50 and 60 for 6 samples 
respectively. 

Bootstrapped Confidence Interval for Mean 

In order to get bootstrap distribution, the "boot" package will be used (Canty & 
Ripley 2016). The "boot" function is used to generate bootstrap distribution for 
specific samples, but this function requires writing simple functions about the 
statistics of interest. 
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> select 

function(x,seed) { set.seed(seed) 
sample<-3ample (x,50) 
return(sample) > 

> mean.func 

function (x, 1} { mean(xd]} } 

The first function is used for the first argument of "boot" function. It is used to 
take the specific sample from the population. The second one is the statistics of 
interest that is "mean" in this example. 

> boot. sampleK-boot (select (population, seed=10) r mean . func, R=2000) 

> boot.3ample2<-boot(select(population,seed=2Q) r mean.func,R=2000) 

> boot.sample3<-boot(select(population,seed=30) r mean.func r R=2000) 

> boot.sample4<-boot(select(population r seed=4Q) r mean.func r R=2000) 

> boot.sample5<-boot(select(population, seed=5Q) r mean.func r R=2000) 

> boot.sample6<-boot(select(population,seed=60) r mean.func f R=2000) 

So we generated six bootstrap distributions with 2,000 resamples. We used the 
"select" and "mean.func" functions within the "boot" functions. Below is the 
bootstrap statistics for the first sample. 

Bootstrap Statistics : 

original bias 3td. error 

tl* 61.33011 -0.02389115 0.3002305 

The value original is the mean of the whole sample while bias is the difference 
between the original mean and the mean of the bootstrapped samples. Standard 
error is the standard deviation of the simulated values. The next step is calculating 
confidence intervals for each original and bootstrapped samples. We used 
the"boot.ci" function to compute confidence intervals for each bootstrapped sample. 

> ci.bootl<-boot.ci(boot.3amplel) > ci.bo 0 t 2 <-bo 0 t.ci(boot.sample2) 

> ci . boot3<-boot. ci (boot. samples) > ci . boot^-boot. ci (boot. sampled) 

> ci.boot5<-boot.ci (boot.samples) > ci.boot6<-boot.ci(boot.sampled) 


Intervals : 

Level Normal 

95^ (59.79, 62.92 ) 


Basic 

(59.75, 62.94 ) 


Level Percentile BCa 

95% (59.72, 62.91 ) (59.79, 62.96 ) 

Calculations and Intervals on Original Scale 


The "basic" title in the R output refers to "basic interval method". In this method 
confidence intervals are estimated by correcting the bootstrap distribution for bias or 
skew. The "Bca" title in R output refers to the "bias corrected and accelerated interval 
method". In the Bca method the bootstrap distribution is corrected for bias and 
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acceleration and the Cl are found at the .025 and .975 quantiles of the corrected 
distribution (Carpenter & Bithell 2000; Banjanovic & Osborne, 2015). 

Comparison of Traditional and Bootstrapped Confidence Intervals for Mean 

The result of the "boot.ci" function produces confidence intervals for four main 
bootstrapped methods discussed earlier. Table 1 presents the comparison of 
confidence intervals of each original random sample and its associated bootstrap 
distributions. 

Table 1 


Confidence Intervals for Original and Bootstrapped Samples 


Sample 

Mean 

Traditional 95% Cl 

Bootstrapped 95% Cl 



Lower Bound 

Upper Bound 

Lower Bound 

Upper Bound 

1 

61.33 

59.76 

62.90 

59.79 

62.92 

2 

60.46 

58.27 

62.65 

58.34 

62.59 

3 

59.73 

57.55 

61.91 

57.54 

61.94 

4 

59.21 

57.33 

61.09 

57.32 

61.08 

5 

61.71 

59.63 

63.89 

59.53 

63.88 

6 

61.24 

59.03 

63.45 

58.97 

63.46 


As seen in Table 1 two methods produce very similar results. The differences are 
at the first or second decimal place. However, it is very important to note that those 
methods will yield very similar results unless the data violates parametric 
assumptions such as normality (Banjanovic & Osborne, 2015). 

Bootstrapped Confidence Intervals for Median 

Median is the observation at the 50th percentile in a set of data ordered from the 
lowest value to the highest value. It is commonly reported and considered a more 
valid definition of center when the frequency distribution of the variable is skewed. 
No simple formula exists for computing confidence intervals for median. According 
to central limit theorem, the number of resampled data sets increasing the 
distribution of the resulting statistic will become approximately normal (Zar, 1999). 
However, using the bootstrapped resampling method, it is possible to calculate the 
confidence interval for median. 

In this example we will calculate the confidence interval for median using two 
different data sets; one is normally distributed and the other is skewed. Next we will 
compare bootstrapped methods with each other. 

We had a normally distributed data set with 1,000,000 observations called 
"population". Now we generate another data set called "population 2" with 
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1,000,000 observations using skewed chi-square distribution. In the previous example 
the population had 100,000 observations. There is no technical reason for increasing 
the number of the observations from 100,000 to 1,000,000. Both are large enough. It is 
aimed to demonstrate to the reader how R could easily generate big data sets. 

> population2<-rchi3q (1000000 r 5) 

> str (population) 

nun [1:1000000] 1.32 2.05 2.62 3.71 3.22 ... 

> | 

First we will compute a confidence interval for median using a normally 
distributed population. We compute bootstrapped distributions and bootstrapped 
confidence intervals using the functions below, previously mentioned in detail. 
Additionally we added a new simple function called "median.func" to calculate the 
median within the "boot" function. 

> median.func 

function(x,i) { median(x[i]} > 

> norm, sample<-boot (select (population r seed=15) f median . func r Et=2 000) 

> norm.sample 

> boot.ci(norm.sample) 

Then we will compute bootstrapped confidence intervals for the median using 
skewed distributed population. 

> skew.samplec-boot(select(population2 f seed=15) r median.func f R=2000) 

> boot.ci(3kew.sample) 

Comparison of Bootstrapping Methods Regarding Distributions of Data Sets 

Table 2 summarizes the confidence intervals for normally distributed and skewed 
data sets regarding bootstrapping methods. 
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Table 2 

Comparison Confidence Intervals Regarding Bootstrapping Methods 


Normally Distributed Skewed Data 

Data 

Methods for Bootstrapping 95% Confidence 95% Confidence 

Interval Interval 



Lower 

Upper 

Lower 

Upper 


Bound 

Bound 

Bound 

Bound 

The normal interval method 

56.80 

59.04 

4.133 

6.071 

The percentile interval method 

56.87 

59.35 

4.081 

5.865 

The basic interval method 

56.66 

59.14 

4.295 

6.078 

Bias corrected and accelerated 
interval method 

56.86 

59.32 

3.740 

5.848 


Table 2 shows that all bootstrapping methods for normally distributed data sets 
produce very similar confidence intervals. The only differences are at the first or 
second decimal place. On the other hand, for skewed data sets bootstrapping 
methods produce different confidence intervals. Therefore, different methods of 
skewed data sets of bootstrapping may produce different results. Before deciding the 
bootstrapping method to be used, their assumptions should be taken into 
consideration. The bias corrected and accelerated interval method requires no 
assumptions about the distribution of the data sets while others do. So it is better to 
consider using this method to compute confidence intervals with skewed data set. 

Bootstrapped Confidence Intervals for Cronbach's Alpha Coefficient 

Cronbach's alpha coefficient is an indicator of reliability that is commonly used, 
especially in psychological tests. In fact, it is the indicator of internal consistency. 
Many researchers use Cronbach's alpha coefficient for a set of items to construct a 
scale. Coefficient alpha (commonly called Cronbach's alpha) was developed by Lee 
Cronbach in 1951 to provide a measure of the internal consistency of a test or scale; it 
is expressed as a number between 0 and 1. Calculating alpha has become common 
practice because it is easier to use than other estimates (e.g. test retest reliability 
estimates) as it only requires one test administration (Tavakol & Dennick, 2011). 
Item-total correlations are the correlation between an item and all other items, where 
the total of the other items is achieved by summing and averaging them (Banjanovic 
& Osborne, 2015). Moreover computing confidence intervals for Cronbach's alpha 
coefficient and item total correlations provides a very good indication of the 
generalizability of the results. 
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To compute traditional item total correlations and Cronbach's alpha coefficient 
the "CTT" package was used (Willse, 2014). In order to compute bootstrapped 
confidence intervals for Cronbach's alpha coefficient and item-total correlations, 
// boot ,/ package has been used. The data is taken from a five-item scale that was 
administered to 300 hundred participants. 

First the item total correlations and Cronbach's alpha coefficient were computed 
with the codes given below, "cr.datal" is the original data set. The object "cr.alp" 
was created to compute the statistics of interest. "cr.alp$alpha" is the Cronbach's 
alpha coefficient and "cr.alp$pBis" is the item total correlations for each of the five 
items. 

> cr . alp<-reliabil±ty (cr .datal, itenial=TRUE) 

> cr.alp$alpha 
[1] 0 .3346353 

> cr.alp$pBis 

[1] 0 .5964466 0 .7006749 0.6642530 0.6159572 0.6021944 


Before bootstrapping 2,000 resamples we were required to write a simple 
function to use within the boot function. "cr. sampl" is the function that might be 
used to compute Cronbach's alpha coefficients and item total correlations for 
bootstrapped resamples. 

> cr.sampl 

function(data, indices) { 
samplec-data[indices, ] 

cr . alpha<-reliability (sample, itenial = TRUE) 
result=with ( cr.alpha,c(alpha,pBis }} 
return(result )> 

I 

Next 2,000 bootstrapped resamples were generated using the boot function given 
below. This process may take 30 minutes to 1 hour depending on the size of the 
original sample and the number of the bootstrapped samples. The values named as 
original in the output are the Cronbach's alpha coefficient (the first row) and item 
total correlations (rows 2 to 5) for the original data set. Bias is the difference between 
the original and the bootstrapped values. Standard error is the standard deviation of 
the simulated values. 

> cror.bach <- boot (data=cr .datal, statistic=cr . sampl, R=2QQ0) 
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Bootstrap Statistics : 

original bias 

tl* 0.3346353 -0.0014236370 
t2* 0.5964466 0.0017199229 

t3* 0.7006749 -0.0001259864 
t4* 0.6642530 -0.0016074003 
t5* 0.6159572 -0.0007566143 
t6* 0.6021944 0.0001311336 


std. error 
0.02290934 
0.05163665 
0.03330456 
0.04733270 
0.05757076 
0.05537665 


The next task is to compute confidence intervals for Cronbach's alpha coefficient 
and item total correlations. The function // boot.ci ,/ was used to calculate those 
intervals, but we needed to write another function including the "boot.ci" function. 
This function is called // computeCT / When writing the code, I was inspired from the 
code written by Banjanovic and Osborne (2015). The first line in the output gives 
confidence intervals for Cronbach's alpha coefficient and the rest of the lines give the 
intervals for item total correlations for each item respectively. I restricted the output 
only with the "percentile" method of bootstrapping discussed above. 

> computeCK-function (alpha) { 

- Conf . Interval<-do . call ( 11 rbind" r sapply{l:6 r FUN = function (i) 

- boot. ci (alpha r t^e= n perc 1T r index=i) $perc [4 : 5] r simplify = FALSE)) 

- return(Conf.Interval ) } 

> computed (cronbach) 

Ml [,2] 

[1,] 0.7351932 0.3749753 
;2,] 0.4932033 0.7001377 
;3,] 0.6206635 0.7704303 
[4 F ] 0.5632592 0.7513773 
;5,] 0.4953220 0.7193234 
[6,] 0.4303203 0.7054129 


Table 3 presents the values and confidence intervals for the statistics of interest. 
Table 3 


Item Total Correlations , Cronbach's Alpha Coefficient, and Related Confidence Intervals 


Items 

Original Item Total 
Correlation 

Bootstrapped confidence 
Intervals 



Lower CI 

Upper CI 

1 

0.60 

0.49 

0.70 

2 

0.70 

0.62 

0.77 

3 

0.66 

0.56 

0.75 

4 

0.62 

0.49 

0.71 

5 

0.60 

0.48 

0.71 

Cronbach Alpha 

0.83 

0.79 

0.87 
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As seen in Table 3, Cronbach's alpha coefficient for the scale is 0.83 and the item 
total correlations vary between 0.60 and 0.70. The confidence intervals for Cronbach's 
alpha is relatively small compared to those for item total correlations (0.79 - 0.87). On 
the other hand, confidence intervals for item total correlations are wide. This may 
lead us to expect that there will be significant variability in the point to estimate for 
item total correlations in other similar samples. The purpose of this study is not to 
construct or adopt a scale, so the results are not discussed in detail. But the point is 
that computing confidence intervals for Cronbach's alpha coefficient and item total 
correlations may guide the researcher while constructing scales. 


Discussion 

Confidence intervals are good indicators of practical significance, unlike p values 
and they also provide more information than p values (Haukas & Lewis, 2005). 
Unfortunately, confidence intervals are rarely reported in academic papers. This is 
because computing confidence intervals are not practical and not possible for some 
statistics. This is why bootstraps methods, which are resampling techniques for 
assessing uncertainty, have become popular. 

In this study the basic principles of bootstrapping and some commonly used 
bootstrapping methods were briefly presented. Then computation of bootstrapping 
methods for mean, median and Cronbach's alpha coefficient were explained using R 
syntax. Furthermore, some comparisons were done. Traditional and bootstrapped 
confidence intervals were compared for mean. Moreover bootstrapped methods 
were compared while calculating median for normally distributed and skewed data 
sets. 

With the help of statistical software today it is easy to compute confidence 
intervals for almost any statistics of interest. R, which is a free and open-sourced 
software, is one of them. This is why in this study R syntax was used as an example 
so that beginners could use the R software to compute confidence intervals. Those 
syntax are not the only solutions but they are just some of the many other possible 
ways to write syntax while computing confidence intervals. 

The results of the comparisons done in this study show that with normally 
distributed data, traditional and bootstrapped confidence intervals are close to each 
other. However, with skewed data sets this may not be so. In these cases the 
bootstrapped methods come into prominence. 

The other important point is deciding the bootstrapped methods to be used when 
computing confidence intervals. Banjanovic and Osborne (2015) present very 
effective strategy to decide the bootstrapped methods to be used. They suggest that 
researchers answer four simple questions. 

1. Is there a formula to estimate the standard error of the statistics? 
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2. Is the distribution symmetrical around the mean of the bootstrapped 
distribution? 

3. Is the distribution normal? 

4. Is the sample estimate a biased estimate of the population statistics? 

The answer for the first question is "yes" for mean but "no" for median. For 
example, for small samples or skewed distributions it may be advised to run 
different methods. If the results differ from each other it may be advised to use the 
bias corrected and accelerated interval method since it has no assumptions about the 
distributions while the others do. Therefore, for the example presented in Table 2 it 
would be better to use the bias corrected and accelerated interval method. However, 
Banjanovic and Osborne (2015) highlighted that bootstrap analysis may not repair 
highly biased samples. This principle should be kept in mind. For in-depth 
information about the assumptions of bootstrapped methods, see Davison & Hinkley 
(1997), Carpenter & Bithell (2000) and Chernick & Labudde (2011). 

Using bootstrapped analysis is also very effective for statistics that already have a 
formula to calculate standard error. Although it is possible to compute standard 
errors for intercept and slope of the regression line in linear regression analysis, 
bootstrapped analysis may produce more accurate standard errors (James et al. 
2014). 

In this study computing bootstrapped methods for mean, median and Cronbach's 
alpha coefficient were illustrated because they are commonly used in social sciences. 
But it is also possible to compute bootstrapped confidence intervals for almost any 
statistics of interest such as effect size, linear regression, logistic regression, factor 
analysis, etc. Each of them may be calculated using "boot" packages illustrated in 
this paper. But this package requires writing a simple chunk of code about the 
statistic of interest, which can be used within the "boot" function. 

Briefly, with help of statistical software like R it is possible to conduct 
bootstrapped analysis. Researchers are now able to compute confidence intervals for 
almost any statistics. Unfortunately, reporting bootstrapped confidence intervals in 
academic journals is very rare. In this study basic steps for calculating bootstrapped 
confidence intervals for some commonly used statistics were illustrated using R 
syntax. Therefore, it is expected that this study guide will enable researchers to 
compute bootstrapped confidence intervals and lead them to report confidence 
intervals in their academic studies. 
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Bootstrap Ornekleme Yonteminin C^e§itli istatistikler igin Giiven 
Araligimn Hesaplanmasmda R Yazilimi ile Kullammi 

(^ali§manm Temeli: Yokluk hipotezine dayali istatistiksel testier istatistiksel 
manidarligi belirlemek igin p degerini kullamr. Akademik dergilerde en gok rapor 
edilen istatistik olan p degeri pratik manidarlik anlammda onemli simrliliklara 
sahiptir. Giiven araligimn p degerine kiyasla daha fazla bilgi sagladigi belirtilebilir. 
Etki biiyiikliigii, giiven araligi ve etki biiyiikliigiine ili§kin giiven araligi pratik 
manidarligm gostergeleri olarak goriilur. Pek gok akademik dergi editorii giiven 
araligi hesaplarmm makalelerde rapor edilmesi gerektigini vurgulamaktadir. Aym 
§ekilde APA 2001 raporunda giiven araligi ve etki buyukliigiiniin raporlanmasimn 
onemi vurgulanmi§tir. Ancak tiim bunlara ragmen akademik makalelerde giiven 
araligimn raporlanma oram gok dii§iiktiir. Bunun bir nedeni ara§tirmacilarm gok 
yaygm rapor edilen p degerinden daha farkli bir istatistigi rapor etme konusundaki 
gekinceleri olabilir. Bu durum ara§tirmacilarm hatali da olsa grubun btiytik 
gogunluguna uyum saglama davram§i sergilemelerine ve bu sayede makalenin kabul 
edilme olasiligma artirmak istemelerine dayali olabilir. Bir diger neden ise bazi 
istatistikler igin giiven araligimn hesaplanmasimn pratik veya miimktin 
olmamasidir. Bootstrap ornekleme yontemi giiven araligi hesaplama siirecinde 
kullamlabilecek giiglti bir istatistiksel yakla§imdir. Temel amag evrenden segilen bir 
orneklem iizerinden yerine koyarak ornekleme yontemi ile yeni orneklemler 
olu§turmaya dayamr. Elde edilen yeni orneklemlere ili§kin istatistikler hesaplanarak 
standart hata ve giiven araligi kestirimi yapilabilir. Bu sayede geleneksel olarak 
standart hata ve giiven araligi tahmini yapilamayan pek gok istatistik igin giiven 
araligimn hesaplanmasma olanak tamr. Yerine koyarak ornekleme yontemi 
kullamldigi igin orijinal orneklemden daha fazla gozlem sayismi igeren orneklemler 
olu§turulabilir. Temelleri 1979 yilmda atilan bootstrap analizinin kullammi bilgisayar 
yazilimlarimn geli§imi ile yaygmlik kazanmi§tir. Yazilimlarm da yardimi ile pratik 
bir §ekilde hesaplanabilecek bu yontem maalesef gok nadir kullamlmakta ve 
ara§tirma makalelerinde rapor edilmemektedir. Bu durumun temel nedeni bootstrap 
yonteminin ve bu siiregte R yazilimimn nasil kullamlacagimn yeteri kadar 
bilinmemesi yatabilir. Alan yazma bakildigmda bootstrap yonteminin R kodlari ile 
agiklandigi gali§malarm smirli oldugu belirlenmi§tir. 

^ali§manm Amaci: Bu gali§mada bootstrap ornekleme yontemine dayali olarak 
yaygm kullamlan bazi istatistikler igin giiven araligi hesaplama stirecinin R kodlari 
ile agiklanmasi amaglanmi§tir. ^ali§mada oncelikli olarak bootstrap ornekleme 
yontemine ili§kin kuramsal bilgiler verilmi§ ve kullamlabilecek farkli bootstrap 
yontemlerine ili§kin agiklamalar yapilmi§tir. Akabinde sosyal bilimlerde yaygm 
olarak kullamlan aritmetik ortalama, ortanca ve Cronbach Alfa igtutarlik katsayisma 
ili§kin bootstrap giiven araligi hesaplama siireci orneklendirilmi§tir. ilk olarak R 
kodlari kullamlarak aritmetik ortalama igin geleneksel giiven araliklari ve ''boot" 
paketinden faydalanarak bootstrap giiven araliklari hesaplanmi§ ve sonuglar 
kar§ila§tmlmi§tir. Daha soma normal ve garpik olan iki dagilim iizerinden ortanca 
igin bootstrap giiven araliklari hesaplanmi§ ve farkli bootstrap yontemleri bu 
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baglamda kar§ila§tirilmi§tir. Son olarak Cronbach Alfa igtutarlik katsayisma ili§kin 
bootstrap giiven araliklari hesaplanmi§ ve orneklendirilmi§tir. 

Kaynak Kamtlari: £ali§ma siirecinde geleneksel giiven araligi hesaplama siirecine 
ili§kin R kodlari sunulmu§tur. Aynca "boot" paketinden faydalamlmi§ ve bu paket 
igerisinde yer alan "boot" ve "boot.ci" fonksiyonlari agiklanmi§tir. Bu paket 
igerisinde yer alan "boot" ve "boot.ci" fonksiyonlari giiven araligi igin hesaplanacak 
her istatistik igin yeni kodlarm yazimmi gerektirmektedir. Bu kodlar da ara§tirma 
igerisinde sunulmu§ ve nasil kullamlacagma yonelik agiklamalarda bulunulmu§tur. 
(^ali§manm ozet boliimun sayfa smirliligi nedeni ile orijinal ingilizce metninde 
sunulan R kodlari ve fonksiyonlar burada verilmemi§tir. Ancak tarti§ma ve sonuglar 
boliimiinde one gikan bulgular ve ilgili yontemin kullammma ili§kin onemli 
agiklamalar sunulmu§tur. 

Tarti§ma ve Sonuglar: Bootstrap ornekleme yontemi uygulama ve yorumlama 
agismdan oldukga pratiktir. R programlama diline ili§kin temel diizeyde bilgiler ile 
bootstrap ornekleme yontemi kolaylikla gergekle§tirilebilir. £ali§ma kapsammda 
gergekle§tirilen kar§ila§tirmada aritmetik ortalama igin geleneksel ve bootstrap 
yontemi ile kestirilen giiven araliklarimn birbirine gok yakm oldugu gozlenmi§tir. 
Ancak bu durumun normal dagilima sahip veri setleri igin boyle oldugu belirtilebilir. 
Bunun yam sira garpik ve normal dagilima sahip veri setleri tizerinden ortanca igin 
kestirilen bootstrap giiven araliklarimn farkli yontemlere gore degi§tigi gbriilmu§tur 
(Normal aralik, yiizdelikli aralik, temel aralik ve yanlilik diizeltmeli aralik). ^arpik 
dagilimlar igin "yanlilik diizeltmeli aralik" yonteminin daha dogru sonug verdigi 
alan yazmda vurgulanmi§tir. Olgek geli§tirme siirecinde kullamlan gergek veri seti 
iizerinde Cronbach Alfa igtutarlik katsayisi ve madde toplam korelasyonlari igin 
hesaplanan bootstrap giiven araliklarimn ise oldukga geni§ oldugu gozlenmi§tir. Bu 
baglamda olgek uyarlama ve geli§tirme siirecinde bootstrap giiven araliklarimn 
hesaplanmasimn ara§tirmacilara onemli boyutta yol gosterecegi ve daha nitelikli 
araglarm geli§tirilmesine katki saglayacagi belirtilebilir. Bootstrap ornekleme 
yonteminin uygulamasi pratik olmakla beraber onemli kuramsal temellere dayandigi 
belirtilebilir. Bu baglamda ilgili okuyucularm ingilizce tarn metinde onerilen 
kaynaklara ula§malan onerilir. Bu gali§manm ara§tirmacilara bootstrap ornekleme 
yontemini kullanma siirecinde yol gostermesi ve ara§tirmacilari giiven araliklarmi 
rapor etmeleri konusunda te§vik etmesi beklenmektedir. 



